Add ability to run spec files in random order #328
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds the following options:
--randomize, which shuffles the list of spec files and
prints out the seed it used to initialize the RNG
--seed SEED, which seeds the RNG to replay an earlier
shuffled test run
For the purposes of getting a test shuffler up and running quickly, I ended up writing my own seedable array shuffler - however, this could be replaced by an existing NPM package if there is one, or moved into its own package.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because jasmine-node doesn't spin up a separate Node process for each spec file, it's possible for global state to get polluted in such a way that specs pass or fail depending on the order in which they're run.
For instance, a spec may accidentally omit the creation of some global config which is required for its class under test - but the spec might still run because a previous spec created the config. Obviously, the best way to deal with this is simply never to use global config, but that's not always practical.
One way to sniff out inadvertent order dependencies is to run spec files in a random order - similar to rspec's
--order randflag.I wrote a relatively quick-and-dirty test order randomizer (attached to pull request) for my team's use, which has already succeeded in finding some inter-spec dependencies in our large set of Jasmine specs.
This adds the following options:
--randomize, which shuffles the list of spec files and prints out the seed it used to initialize the RNG--seed SEED, which seeds the RNG to replay an earlier shuffled test runIs this something that the jasmine-node project would be interested in? If so, I'm happy to massage it into commit-able shape, add tests etc.